Time-of-flight (TOF) systems that provide a measure of distance (Z) from the system to a target object without depending upon luminosity or brightness information obtained from the target object are known in the art. See for example U.S. Pat. No. 6,323,942 entitled CMOS-Compatible Three-Dimensional Image Sensor IC (2001), assigned to Canesta, Inc., now of Sunnyvale, Calif. TOF systems according to the '942 patent emit optical energy and determine how long it takes until at least some of that energy reflected by a target object arrives back at the system to be detected. Emitted optical energy traversing to more distant surface regions of a target object before being reflected back toward the system will define a greater TOF than if the target object were closer to the system. If the roundtrip TOF time is denoted t1, then the distance between target object and the TOF system is Z1, where Z1=t1·C/2, where C is velocity of light. Such systems can acquire both luminosity date (signal amplitude) and TOF distance, and can realize three-dimensional images of a target object in real time.
A more sophisticated TOF system is exemplified by U.S. Pat. Nos. 6,515,740 (2003) and 6,580,496 (2003) respectively Methods and Systems for CMOS-Compatible Three-Dimensional Imaging Sensing Using Quantum Efficiency Modulation, assigned to Canesta, Inc., now of Sunnyvale, Calif. FIG. 1A depicts an exemplary phase-shift detection system 100 according to the '740 or the '296 patents, a system in which TOF is determined by examining relative phase shift between transmitted light signals and light signals reflected from the target object. Detection of the reflected light signals over multiple locations in the system pixel array results in measurement signals that are referred to as depth images. The depth images represent a three-dimensional image of the target object surface.
Referring to FIG. 1A, TOF system 100 includes a two-dimensional array 130 of pixel detectors 140, each of which has dedicated circuitry 150 for processing detection charge output by the associated detector. In a typical application, array 130 might include 100×100 pixels 230, and thus include 100×100 processing circuits 150. IC 110 may also include a microprocessor or microcontroller unit 160, memory 170 (which preferably includes random access memory or RAM and read-only memory or ROM), a high speed distributable clock 180, and various computing and input/output (I/O) circuitry 190. Among other functions, controller unit 160 may perform distance to object and object velocity calculations.
Under control of microprocessor 160, a source of optical energy 120 is periodically energized via exciter 115, and emits optical energy via lens 125 toward an object target 20. Typically the optical energy is light, for example emitted by a laser diode or LED device 120. Some of the emitted optical energy will be reflected off the surface of target object 20, and will pass through an aperture field stop and lens, collectively 135, and will fall upon two-dimensional array 130 of pixel detectors 140 where an image is formed. In some implementations, each imaging pixel detector 140 captures time-of-flight (TOF) required for optical energy transmitted by emitter 120 to reach target object 20 and be reflected back for detection by two-dimensional sensor array 130. Using this TOF information, distances Z can be determined. Advantageously system 100 can be implemented on a single IC 110, without moving parts and with relatively few off-chip components.
Typically optical energy source 20 emits preferably low power (e.g., perhaps 500 mW peak) periodic waveforms, producing optical energy emissions of known frequency (perhaps 30 MHz to a many hundred MHz) for a time period known as the shutter time (perhaps 10 ms). Optical energy from emitter 120 and detected optical energy signals within pixel detectors 140 are synchronous to each other such that phase difference and thus distance Z can be measured for each pixel detector. The detection method used is referred to as homodyne detection in the '740 and '496 patents. Phase-based homodyne detection TOF systems are also described in U.S. Pat. No. 6,906,793, Methods and Devices for Charge Management for Three-Dimensional Sensing, assigned to Canesta, Inc., assignee herein.
The optical energy detected by the two-dimensional imaging sensor array 130 will include light source amplitude or intensity information, denoted as “A”, as well as phase shift information, denoted as 4. As depicted in exemplary waveforms in FIGS. 1B and 1C, the received phase shift information (FIG. 1C) varies with TOF and can be processed to yield Z data. For each pulse train of optical energy transmitted by emitter 120, a three-dimensional image of the visible portion of target object 20 is acquired, from which intensity and Z data is obtained (DATA). As described in U.S. Pat. Nos. 6,515,740 and 6,580,496 obtaining depth information Z requires acquiring at least two samples of the target object (or scene) 20 with 90° phase shift between emitted optical energy and the pixel detected signals. While two samples is a minimum figure, preferably four samples, 90° apart in phase, are acquired to permit detection error reduction due to mismatches in pixel detector performance, mismatches in associated electronic implementations, and other errors. On a per pixel detector basis, the measured four sample data are combined to produce actual Z depth information data. Further details as to implementation of various embodiments of phase shift systems may be found in U.S. Pat. Nos. 6,515,740 and 6,580,496.
FIG. 1C is taken from U.S. Pat. No. 6,580,496 and depicts exemplary fixed phase delay (VPD) homodyne quantum efficiency (QE) detection. As described herein, such detection involves homodyning (or mixing) detected signal S2 (generated from emitter 120) received at a pixel 140, with a reference signal derived from the clock circuit 180 controlling generator 115 and light source 120. Without loss of generality and for ease of explanation, it will be assumed that all clocks ĉ and signals ŝ are zero mean ĉ=0 and ŝ=0, e.g., an average duty cycle of 50%.
In a phase based TOF system 100, the Z distance of a target object is proportional to the phase delay of the modulated signal ŝp(t) (or S2 in FIG. 1A) received at a pixel 140 in pixel array 130. Thus by computing the phase delay, the distance Z between the TOF system and a surface point on target object 20 can be readily computed. Phase is computed by integrating the cross product of the pixel signal ŝp(t) with a reference clock c(t) that is synchronized with the clock system 180 (see FIG. 1A) that commands optical energy source 120. As noted above, phase is computed from multiple, preferably four, samples 90° apart. Each such sample may be considered to be pixel output R, where the pixel integrates the modulated light over a period T:
  R  =            ∫      0      T        ⁢                            c          1                ⁡                  (          t          )                    ·                        s          p                ⁡                  (          t          )                    ·                          ⁢                        ⅆ          t                .            
As will be described later with respect to FIGS. 2A and 2B, if two systems 100, e.g., system 100-A, 100-B are operating in relatively close proximity, the pixel in system 100-A may also integrate a signal ŝp(t) from optical energy emitted by system 100-B, and vice versa. With respect to the pixel in system 100-A, total pixel response will now be
      R    total    =            ∫      0      T        ⁢                            c          1                ⁡                  (          t          )                    ·              (                                            s              p                        ⁡                          (              t              )                                +                                    s              b                        ⁡                          (              t              )                                      )            ·                          ⁢                        ⅆ          t                .            To reduce impact of the signal generated by system 100-B upon system 100-A, one must minimize
      R    error    =            ∫      0      T        ⁢                            c          1                ⁡                  (          t          )                    ·                        s          b                ⁡                  (          t          )                    ·                          ⁢                        ⅆ          t                .            Since ŝb(t) is synchronized with clock c2 (e.g., clock 180) of system 100-B, inter-system interference is (statistically) proportional to
      ∫    0    T    ⁢                    c        1            ⁡              (        t        )              ·                  c        2            ⁡              (        t        )              ·                  ⁢          ⅆ      t      where is the integration interval. The above integration may be said to define a cross-correlation product P12, whose magnitude the present invention seeks to minimize. A self-correlation product P11 may be defined as
      ∫    0    T    ⁢                    c        1            ⁡              (        t        )              ·                  c        1            ⁡              (        t        )              ·                  ⁢                  ⅆ        t            .      It is understood that P11, P12 are time-varying samples that represent statistical characteristics of the clock signal.
FIG. 1D is useful in better understanding homodyne detection in a phase-based TOF system. FIG. 1D depicts a portion of array 130 on IC 110 (see FIG. 1A), where 140-1, 140-N denote two differential pixel detectors, and 150-1, 150-N denote respective associated pixel electronics. FIG. 1D is similar to what is described with respect to the fixed phase delay embodiment of FIG. 10 in U.S. Pat. No. 6,580,496, entitled Systems for CMOS-Compatible Three-Dimensional Image Sensing Using Quantum Efficiency Modulation, or in U.S. Pat. No. 7,906,793, entitled Methods and Devices for Charge Management for Three-Dimensional Sensing, both patents assigned to Canesta, Inc., assignee herein. In FIG. 1D, detection-generated photocurrent from each QE-modulated differential pixel detector, e.g., 140-1, is differentially detected (DIF. DETECT) and differentially amplified (AMP) to yield signals B·cos(φ), B·sin(φ), where B is a brightness coefficient, A fixed 0° or 90° phase shift delay (DELAY) is switchably insertable responsive to a phase select control signal (PHASE SELECT). The configuration also includes a 180° inverting stage. Homodyne mixing occurs using QE modulation to derive phase difference between transmitted and received signals (see FIGS. 1B, 1C), and to derive TOF, among other data. A more detailed description of homodyne detection in phase-based TOF systems is found in the '496 patent. Although sinusoidal type periodic waveforms are indicated in FIG. 1D, non-sinusoidal waveforms may instead be used.
As noted, if two or more TOF systems 100, e.g., 100-A, 100-B, are operated in relatively close proximity, system 100-A may detect optical energy emitted by system 100-B (in addition to detecting optical energy emitted by system 100-A) and vice versa. Such scenarios are exemplified by FIGS. 2A and 2B, where phase-based TOF systems are deployed in motor vehicles, and where inter-system interference can corrupt the desired DATA. In FIG. 2A, motor vehicle 200-A is provided with a phase-based TOF system 100-A, and motor vehicle 200-B is provided with a similar phase-based TOF system 100-B. In FIG. 2A, radiation S1-A is emitted by system 100-A, and radiation S1-B is emitted by system 100-B.
In an actual system, there would be several emitters (such as emitters 120 in FIG. 1A) disposed at various locations on each vehicle so as to emit optical energy to the right, to the left, forward, and to the rear. (Similarly there would be a number of detector arrays, such as 130 in FIG. 1A, disposed to detect target-object reflected emitted radiation.) If radiation S1-A emitted from system 100-A reflects off a target object, e.g., vehicle 100-B, system 100 can determine the distance Z between the two vehicles. DATA from system 100 can be used to signal a warning to the operator of vehicle 100-A, and/or command the steering system of system 100-A to take safe evasive action, and/or to intelligently deploy air bag(s) within vehicle 100-A is dynamic Z values indicated a collision is unavoidably imminent. However if the target object is another vehicle, vehicle 100-B, equipped with a similar system, 100-B, optical energy detected by system 100-A may include reflected S1-B optical energy in addition to reflected S1-A optical energy. The resultant inter-system interference or interaction can result in erroneous or corrupt DATA from each system. In an application such as shown in FIG. 2A, motor vehicle collisions could result from reliance upon corrupted DATA.
FIG. 2B depicts another example of vehicular inter-system interference. In this example, two vehicles 200-A, 200-B are each backing up towards a wall 210. Normally system 100-A in vehicle 200-A would detect emitted radiation reflected by the surface of a target object, here wall 210. Valid DATA from system 100-A could alert the operator of vehicle 200-A if distance Z to the wall is suddenly too low, e.g., vehicle 100-A is inadvertently about to hit the wall. But if vehicle 100-B has a similar system 100-B, interference between the two systems can result in each system determining erroneous data, e.g., corrupt DATA. The result might be that each vehicle would be allowed to collide with wall 210.
While FIGS. 2A and 2B depict prior art phase-based TOF system 100 used in motor vehicle applications, such system also found use in other applications. For example, it is known in the art to employ TOF systems to determine distance Z between a system that may be phase-based such as system 100, and locations on a virtual input device. The virtual input device can be a virtual keyboard where locations thereon represent virtual keys “contacted” by a user's finger or stylus. See for example U.S. Pat. No. 6,614,422 Method and Apparatus for Entering Data Using a Virtual Input Device, assigned to Canesta, Inc., assignee herein. It will be appreciated that if more than one virtual input system were used in close physical proximity to another such system, active light emitted by one system could interfere with detection of reflected light emitter by the other system, and vice versa. In a virtual keyboard application, the result would be misinterpreted keystrokes, e.g., many typographical errors.
As the popularity of phase-based TOF systems increases, the likelihood of inter-system interference from adjacent systems will increase. In a virtual keyboard application, inter-system errors would merely result in mistyped characters. But in vehicle safety applications, or in so-called industrial light curtain safety applications, inter-system errors could result in substantial damage to lives and property.
In TOF systems, pixel output R is usually determined to a precision better than 1%. Thus it is desirable to maintain magnitude of Rerror such that the absolute magnitude of the interference quotient Rerror/Rtotal<1%. But in an application such as shown in FIG. 2B, if system 100-B is three times closer to wall 210 than is system 100-A, and if both system output equal optical power, the signal from system 100-B will be about ten times brighter than that from system 100-A. If under these conditions it is desired that Rerror/Rtotal<1% (absolute magnitude), then assuming equal optical power, the result suggests that approximately the absolute magnitude of P12/P11<10−3. Stated different, the Rerror/Rtotal quotient represents the ratio between two brightness amplitudes that may vary widely in magnitude, especially if one system illuminates a target object with say ten times more intensity, e.g., perhaps because the target object is closer. In implementing phase-based TOF systems 100, convention wisdom has dictated that clock system 180 should be a high quality clock, characterized by stable frequency and low noise characteristics.
Oscillator noise including modeling of naturally occurring noise in oscillators has been extensively studied, and numerous references are widely available, including treatises and other research authored by Behzad Razavi. By carefully shaping and controlling naturally occurring oscillator noise, e.g., thermal noise 1/KtC, shot noise, 1/f noise, the correlation product P12 can be minimized without substantially changing system accuracy. It is known in the art to reduce the impact of such oscillator noise sources in applications by careful oscillator design and use of techniques including phase-lock loops (PLL), delay lock loops (DLL), etc. Exemplary specifications for such prior art clock systems might be clock frequency stability on the order of 10 to 30 ppm at a nominal clock frequency of about 0.5 GHz to perhaps about 1 GHz, and clock phase stability on the order of 1/1000, or in terms of a Fourier analysis, a 100 dbc at a 100 KHz deviation.
Attempts have been made in the prior art to address the inter-system interference problem noted above with respect to phase-based TOF systems. For example EP1647839A2 assigned to PMD, and entitled Enffernungsnmess-vorrichtung zum Bestimmen der Enffernung Zwischen Einem Objekt & der Vorrich-tung, translated as Method and Distance Measuring Device for Determining the Distance Between an Object and the Device seeks to change the clock operating frequency at each frame of phase capture, e.g., between integration periods, but apparently changes nothing within integration periods. But implementing this approach has many problems. For example, in many phase-based TOF systems, multiple phase captures are combined to produce a frame, and multiple frames may share multiple. If each phase capture were of a different clock frequency, meaningfully combining their respective data to produce a frame of depth data is mathematically very complex. Additionally, a TOF system is usually calibrated to operate at one particular average frequency faverage. Different operating frequencies can result in multiple calibration tables, one for each frequency. Further, it will always be possible that the frequencies of two TOF systems may be the same during a capture, in which case the systems are susceptible to the inter-system errors noted herein. Stated differently, attempting to randomize inter-system interference by changing the frequency between integration periods is somewhat impractical and ineffective.
What is needed then is a method and system to minimize the likelihood of such intra-system interference in a phase-based TOF system. Preferably such method and system should avoid using clocking systems that are difficult or expensive to implement, or that cannot be implemented using CMOS-compatible techniques.
The present invention provides a method and system to reduce the likelihood of intra-system interference from adjacent phase-based TOF systems.